Can AI Assistants Know What They Don't Know?

Recently, AI assistants based on large language models (LLMs) show surprising performance in many tasks, such as dialogue, solving math problems, writing code, and using tools. Although LLMs possess intensive world knowledge, they still make factual errors when facing some knowledge intensive tasks, like open-domain question answering. These untruthful responses from the AI assistant may cause significant risks in practical applications. We believe that an AI assistant's refusal to answer questions it does not know is a crucial method for reducing hallucinations and making the assistant truthful. Therefore, in this paper, we ask the question "Can AI assistants know what they don't know and express them through natural language?" To answer this question, we construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions, based on existing open-domain question answering datasets. Then we align the assistant with its corresponding Idk dataset and observe whether it can refuse to answer its unknown questions after alignment. Experimental results show that after alignment with Idk datasets, the assistant can refuse to answer most its unknown questions. For questions they attempt to answer, the accuracy is significantly higher than before the alignment.Comment: Work in progres

Deformation and failure characteristics of gas drainage drilling-reaming coal mass in non-uniform stress field

The deformation and failure characteristics of coal around gas drainage boreholes in deep soft and low permeability coal seams affect coal seam gas pre-drainage. Based on the condition of non-uniform stress field, the mechanical model of borehole disturbed coal mass was developed, the analytical solutions of stress, strain and displacement in the damaged zone, plastic zone and elastic zone of borehole disturbed coal mass were deduced, the influence law of factors such as lateral pressure coefficient, load condition, cohesion and hole expanding behavior on the “three zone” distribution of disturbed coal mass were analyzed, and the reliability of the theoretical model was verified through engineering examples. The results show that under the condition of non-uniform stress field, the plastic zone and damaged zone of disturbed coal mass are elliptical distribution. With the increase of lateral pressure coefficient, the length of the upper and lower wings of the plastic zone and damaged zone of disturbed coal mass becomes larger and larger, and the radius of the plastic zone and damaged zone in the direction of smaller stress is greater than the radius of the two zones in the direction of large stress. The radius of plastic zone and damaged zone of coal mass increases with the increase of vertical load, and decreases with the increase of initial cohesion and residual cohesion. The influence of vertical load on its shape can be ignored. When the borehole diameter is expanded from 0.1 m to 0.5 m, the coal mass 0−1.0 m away from the borehole center produces a strong disturbance, the coal mass 1.0−4.6 m produces a weak disturbance, and the coal mass after 4.6 m has almost no influence. Through the field example of No.16032 bottom pumping roadway hydraulic reaming in the Guhanshan coal mine, it is observed that the disturbed coal mass in the reaming section has a high degree of damage. Based on the coal output, the reaming diameter is deduced to be 1.5 m, and then the deformation and damage characteristics of drilling reaming coal mass are obtained through theoretical calculation and numerical simulation respectively. The two are in good agreement, so as to verify the reliability of the theoretical model

Predicting Academic Performance: A Systematic Literature Review

The ability to predict student performance in a course or program creates opportunities to improve educational outcomes. With effective performance prediction approaches, instructors can allocate resources and instruction more accurately. Research in this area seeks to identify features that can be used to make predictions, to identify algorithms that can improve predictions, and to quantify aspects of student performance. Moreover, research in predicting student performance seeks to determine interrelated features and to identify the underlying reasons why certain features work better than others. This working group report presents a systematic literature review of work in the area of predicting student performance. Our analysis shows a clearly increasing amount of research in this area, as well as an increasing variety of techniques used. At the same time, the review uncovered a number of issues with research quality that drives a need for the community to provide more detailed reporting of methods and results and to increase efforts to validate and replicate work.Peer reviewe

Retrospective evaluation of whole exome and genome mutation calls in 746 cancer samples

Funder: NCI U24CA211006Abstract: The Cancer Genome Atlas (TCGA) and International Cancer Genome Consortium (ICGC) curated consensus somatic mutation calls using whole exome sequencing (WES) and whole genome sequencing (WGS), respectively. Here, as part of the ICGC/TCGA Pan-Cancer Analysis of Whole Genomes (PCAWG) Consortium, which aggregated whole genome sequencing data from 2,658 cancers across 38 tumour types, we compare WES and WGS side-by-side from 746 TCGA samples, finding that ~80% of mutations overlap in covered exonic regions. We estimate that low variant allele fraction (VAF < 15%) and clonal heterogeneity contribute up to 68% of private WGS mutations and 71% of private WES mutations. We observe that ~30% of private WGS mutations trace to mutations identified by a single variant caller in WES consensus efforts. WGS captures both ~50% more variation in exonic regions and un-observed mutations in loci with variable GC-content. Together, our analysis highlights technological divergences between two reproducible somatic variant detection efforts

Wintering bird communities in newly-formed wetland in the Yangtze River estuary

We compared wintering bird communities and their habitats among three shoals at Jiuduansha, a newly-formed wetland in the Yangtze River estuary. The highest species richness and diversity were recorded in Shangsha, which is the highest shoal, and the highest abundance and lowest species diversity were recorded in Xiasha, which is the lowest shoal. Shangsha had the largest abundance of perching birds whereas Xiasha was the most abundant in waterbirds. Bird assemblages showed different associations with the different habitat types-perching birds were favored by reed (Phragmites australis) communities, shallow water foragers and dabbling ducks preferred sea-bulrush (Scirpus mariqueter) communities, and moist-soil foragers and gulls showed a preference for bare intertidal zones. All bird assemblages, however, avoided the smooth cordgrass (Spartina alterniflora) communities, which are dominated by an alien invasive plant. The composition of avian communities was related to habitat types at the three shoals. Our results suggest that the newly-formed tidelands can provide suitable habitats for waterbirds and that the lower tidelands can attract more waterfowl than the higher tidelands. Because the shoal with low species diversity could have exclusive bird species, conservation efforts should not concentrate only on the area with high species diversity. The estuarine wetlands should be considered as a whole when conservation strategies are designed. The alien invasive plant should, moreover, be effectively controlled, to provide suitable habitats for birds